---
title: 'Using profile tcprtt'
weight: 20
description: >
  Analyze TCP connections through an Round-Trip Time (RTT) distribution
---

The profile tcprtt gadget generates a histogram distribution of the TCP
connections' Round-Trip Time (RTT). The RTT values used to create the histogram
are collected from [the smoothed
RTT](https://elixir.bootlin.com/linux/v5.11.22/source/include/linux/tcp.h#L258)
information already provided by the Linux kernel for the TCP sockets.

The histogram considers only the TCP connections that have been already
established, so it does not take into account the connection phase (3-way TCP
Handshake). If it is what you are looking for, please check the latency
information the trace tcpconnect gadget provides. See further information
[here](../trace/tcpconnect.md#calculating-the-latency-of-a-connection).

By default, the profile tcprtt gadget generates one single histogram per host,
and one per node in case of Kubernetes. However, it also provides multiple ways
to analyze specific connections. For instance, we can generate multiple
histograms separated by local IP addresses to explore all the connections
generated by each local address. And the same can be done for remote IP
addresses. In addition, it is also possible to filter by a specific local and/or
remote address to isolate the analysis.

### On Kubernetes

First of all, let's start the gadget on a terminal:

```bash
kubectl gadget profile tcprtt
```

In another terminal, create a server using nginx:

```bash
kubectl create service nodeport nginx --tcp=80:80
kubectl create deployment nginx --image=nginx
```

And then, create a pod to generate some traffic with the server:

```bash
$ kubectl run -ti --privileged --image wbitt/network-multitool myclientpod -- bash
# curl nginx
# curl nginx
```

If we move back to the first terminal and stop the gadget, it will generate the histograms:

```bash
All Addresses = ****** [AVG 1211.066824]
        µs               : count    distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 114      |***********                             |
        16 -> 31         : 397      |****************************************|
        32 -> 63         : 182      |******************                      |
        64 -> 127        : 49       |****                                    |
       128 -> 255        : 48       |****                                    |
       256 -> 511        : 107      |**********                              |
       512 -> 1023       : 108      |**********                              |
      1024 -> 2047       : 86       |********                                |
      2048 -> 4095       : 31       |***                                     |
      4096 -> 8191       : 111      |***********                             |
      8192 -> 16383      : 28       |**                                      |
     16384 -> 32767      : 11       |*                                       |
```

Take into account that the generated histogram considers all TCP connections for
each node, not only the ones we established. Notice that we used a cluster with
a single node for this guide.

So, let's repeat the test but this time filtering by remote address so that we
can analyse the traffic we are generating toward our nginx service:

```bash
$ kubectl get service nginx -o jsonpath={.spec.clusterIP}
10.0.38.234
$ kubectl gadget profile tcprtt --raddr 10.0.38.234
All Addresses = ****** [AVG 2087.000000]
        µs               : count    distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 0        |                                        |
       256 -> 511        : 0        |                                        |
       512 -> 1023       : 0        |                                        |
      1024 -> 2047       : 3        |****************************************|
      2048 -> 4095       : 3        |****************************************|

```

Now, let's use the [network
emulator](https://wiki.linuxfoundation.org/networking/netem) to introduce some
random delay to the packets and increase indirectly the RTT:

```bash
# tc qdisc add dev eth0 root netem delay 50ms 50ms 25%
# curl nginx
# curl nginx
```

Now the average RTT value of the new histogram is clearly higher:

```bash
$ kubectl gadget profile tcprtt --raddr 10.0.38.234
All Addresses = ****** [AVG 68973.833333]
        µs               : count    distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 0        |                                        |
       256 -> 511        : 0        |                                        |
       512 -> 1023       : 0        |                                        |
      1024 -> 2047       : 0        |                                        |
      2048 -> 4095       : 0        |                                        |
      4096 -> 8191       : 0        |                                        |
      8192 -> 16383      : 0        |                                        |
     16384 -> 32767      : 0        |                                        |
     32768 -> 65535      : 3        |****************************************|
     65536 -> 131071     : 3        |****************************************|

```

### With `ig`

Start the profile tcprtt gadget on a first terminal:

```bash
sudo ig profile tcprtt
```

Then, start a container and download a web page:

```bash
$ docker run -ti --rm --cap-add NET_ADMIN --name=netem wbitt/network-multitool -- /bin/bash
# wget 1.1.1.1
```

Moving back to the first terminal and stopping the gadget, it will generate the histograms:

```bash
All Addresses = ****** [AVG 2343.510333]
        µs               : count    distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 25       |                                        |
         8 -> 15         : 226      |**                                      |
        16 -> 31         : 777      |********                                |
        32 -> 63         : 1532     |****************                        |
        64 -> 127        : 2822     |*******************************         |
       128 -> 255        : 2254     |************************                |
       256 -> 511        : 3305     |************************************    |
       512 -> 1023       : 2863     |*******************************         |
      1024 -> 2047       : 1284     |**************                          |
      2048 -> 4095       : 1456     |****************                        |
      4096 -> 8191       : 3612     |****************************************|
      8192 -> 16383      : 167      |*                                       |
     16384 -> 32767      : 0        |                                        |
     32768 -> 65535      : 14       |                                        |
     65536 -> 131071     : 75       |                                        |
    131072 -> 262143     : 0        |                                        |
    262144 -> 524287     : 0        |                                        |
    524288 -> 1048575    : 8        |                                        |
```

This histogram represents the distribution of the RTT for all the TCP
connections established in the host and not only the TCP connections we
established in the test container.

Let's repeat the test but this time filtering by remote address so that we can
analyse the traffic we are generating:

```bash
$ sudo ig profile tcprtt --raddr 1.1.1.1
All Addresses = ****** [AVG 7359.357143]
        µs               : count    distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 0        |                                        |
       256 -> 511        : 0        |                                        |
       512 -> 1023       : 0        |                                        |
      1024 -> 2047       : 0        |                                        |
      2048 -> 4095       : 0        |                                        |
      4096 -> 8191       : 9        |****************************************|
      8192 -> 16383      : 5        |**********************                  |
```

Now, let's introduce some random delay to the packets to increase indirectly the
RTT using the [network
emulator](https://wiki.linuxfoundation.org/networking/netem):

```bash
# tc qdisc add dev eth0 root netem delay 50ms 50ms 25%
# wget 1.1.1.1
```

And, regenerate the histogram to see the change in the RTT:

```bash
$ sudo ig profile tcprtt --raddr 1.1.1.1
All Addresses = ****** [AVG 72278.307692]
        µs               : count    distribution
         0 -> 1          : 0        |                                        |
         2 -> 3          : 0        |                                        |
         4 -> 7          : 0        |                                        |
         8 -> 15         : 0        |                                        |
        16 -> 31         : 0        |                                        |
        32 -> 63         : 0        |                                        |
        64 -> 127        : 0        |                                        |
       128 -> 255        : 0        |                                        |
       256 -> 511        : 0        |                                        |
       512 -> 1023       : 0        |                                        |
      1024 -> 2047       : 0        |                                        |
      2048 -> 4095       : 0        |                                        |
      4096 -> 8191       : 0        |                                        |
      8192 -> 16383      : 0        |                                        |
     16384 -> 32767      : 0        |                                        |
     32768 -> 65535      : 0        |                                        |
     65536 -> 131071     : 13       |****************************************|
```

We can see how the average RTT passed from 7359.357143 to 72278.307692.
